Recommendations for Research and Clinical Implementation of Ambulatory Assessment, Mood Monitoring, Digital Phenotyping, and Remote Measurement Technology in Mood Disorders: Synthesis of Systematic Review Findings

Introduction

This synthesis of systematic reviews evaluates medium- or long-term (3 months or more) ambulatory assessment and mood monitoring protocols in depression and bipolar disorder (BD). It provides evidence-based recommendations for clinical and research use based on effectiveness, performance, and user experience. These recommendations serve as a key blueprint for the design and implementation of future ambulatory assessment protocols.

First, we define the different types of ambulatory assessment discussed in this review and the different use cases of these types. Ambulatory assessment includes ecological momentary assessment (EMA—more intensive self-report eg, multiple times per day [1,2]), remote measurement technology (eg, wearables to passively collect data in the background), and mood monitoring or mood tracking. This review assesses mood monitoring studies, and some of these studies also fall under EMA, remote measurement technology, and ambulatory assessment studies, but there is often overlap in their definitions. Mood monitoring can be used as a research measure (both in randomized controlled trials [RCTs] and nonrandomized studies) and as an intervention (both in RCTs and nonrandomized studies)—both in potentially highly personalized ways that offer improvements in efficiency, flexibility, and usability [3,4]. Mood monitoring can be active or passive. Passive monitoring does not require input from the user to collect data and operates in the background. Passive monitoring is less intrusive and occurs automatically but often lacks granularity [5] because the interpretation of these measures requires additional context (eg, if the individual is experiencing agitation or just taking a walk in the case of step counters). These measures of objective behavior or physiology, however, do not explore the subjective experience of mood [6]. Active monitoring (requiring the participant to actively input data) takes effort on the part of the participant, adds granularity to passive measures, and can be completed when convenient for the participant [7]. EMA is perceived as more intrusive because it requires responses in the moment, sometimes when it is not convenient. EMA does not provide a snapshot over time unless the data are aggregated—it captures subjective experience but does not fully explore the content because it is brief and occurs in the moment [1,8]. These distinct tools all have the potential to complement each other when used in the right way. Furthermore, using modern techniques, it is common for EMA to assess multiple aspects of experience at the same time, for example, location, social context, activities, mood, and psychological symptoms in a single survey [9]. This density of sampling determines content coverage, and if participants are sampled several times a day, a wide range of information can be collected, not just information related to mood and psychological symptoms [8]. In this study, we evaluate and discuss these different use cases.

This review is particularly important as the interest in ambulatory assessment approaches in mental health has increased substantially in recent years. This is due to the development of new technologies that may be key in delivering new mental health treatments and collecting large volumes of research data that can potentially capture the dynamic fluctuations in psychopathology and behavior that traditional measures or interventions can struggle to capture [10]. These fluctuations are associated with mental relapse, recurrence, wellness, reliable recovery, distress, and risk-taking behaviors [11]. Ambulatory assessment offers the possibility of detecting mood instability earlier and more continuously than traditional methods. The measurement of these fluctuations may improve personalized prediction and prognosis as well as treatment outcomes via digital phenotyping [12]. Ambulatory assessment approaches may also be more convenient. For example, while they may avoid the felt stigma of certain engagements with mental health services (eg, attending a clinic) and avoid language-based complications (eg, translation), they do, however, often require significant motivation on the part of the participant.

While ambulatory assessment and mood monitoring approaches hold promise, current evidence reveals mixed performance, uncertain efficacy, and complex user experiences [13]. On the one hand, mood tracking via new technologies (such as wearable devices and smartphone sensors) is broadly liked by people with mood disorders and is often used to support well-being [14,15]. On the other hand, there are concerns within the field regarding the implementation of these technologies [16,17]. Some of these concerns include the performance of the measures (eg, do they measure what they claim to?), their usability and acceptability, the possibility of adverse events and negative psychological impacts, and whether they exert any direct effect on mood or not (eg, will they make effective interventions for people with mental disorders?) [13-15]. We sought to provide key answers to these concerns by reviewing the data—specifically focusing on studies that were closer to clinical implementation (eg, with follow-up periods of a minimum of 3 months).

Here, we summarize the results of a series of systematic reviews and provide evidence-based recommendations for the development of future ambulatory assessment protocols or mood monitoring interventions. We treat these 2 functions—one as the gathering of data for research assessment and the other as the gathering of data to deliver a personalized intervention (eg, just-in-time adaptive interventions [18])—as 2 distinct purposes; therefore, the recommendations or considerations are separate and different in nuanced ways. In this review, we included studies that have assessed mood at least weekly, with most ambulatory assessment protocols using daily, multiple-times-per-day, or continuous assessment. The frequency of assessment depended on the purpose of the study and the behavior assessed (eg, continuous step count monitoring vs visual analog scale of mood).

The majority of the reviews reported here have been published, and there was not sufficient space in these studies to fully detail these important recommendations to the field—both for future interventions and research measures. We present this as a separate paper to fully discuss the implications of our research for the future development of these technologies. This is a common approach that expands on already published data by providing a broader expert consensus, which provides important context to the findings of the 7 individual papers. This does not, however, include any new unpublished data (or data currently in peer review—one paper).

Methods

Recommendations and Grading of the Evidence

Evidence-based recommendations and grading of the strength of the evidence for each recommendation were developed up by a multidisciplinary group involving psychiatrists, methodologists, and service users who were involved in the original design, data analysis, and subsequent interpretation of the reviews. We used GRADE (Grading of Recommendations Assessment, Development and Evaluation) to assess the quality of the evidence and the strength of the recommendations (high, moderate, low, or very low) [19].

Methodology

This paper itself is not a review but synthesises the findings of a series of reviews, the methodology of which is discussed here.

We include the eligibility criteria here as these are important for appraising the recommendations we make in this paper. We included studies that met the following criteria: (1) quantitative studies including self-monitoring, EMA, or repeated symptom assessment in people with depression or BD using an interventional design over a minimum period of 3 months, with symptoms rated weekly at a minimum. We set this minimum period of 3 months of use to include studies that were closer to real-world implementation and because we reasoned that 3 months of use was the minimum requirement for many people with BD or depression to observe a significant change in mood. Three months is also a frequent primary outcome assessment point for many key interventional studies. (2) Qualitative studies exploring user perspectives on self-monitoring or ambulatory assessment or repeated symptom assessment in people with depression or BD. We defined depression as a current or previous clinical diagnosis, a self-reported diagnosis, or meeting research criteria for a depressive disorder. Self-reported depressive symptoms and depression in the context of BD or depression were excluded, as the user experience was theorized to be distinct. We included actual use and hypothetical use of ambulatory assessment or self-monitoring for depressive symptoms. The studies could be published in any language and could involve digital or nondigital methods.

Results

RCTs and nonrandomized studies used ambulatory assessment or mood tracking as an outcome or an intervention. Qualitative studies explored the user experience of ambulatory assessment or mood tracking protocols, or used coproduction or implementation frameworks to improve the usability and acceptability of the protocol. Table 1 outlines the overall findings from all the systematic reviews.

The search identified 23,515 papers. A total of 111 papers met the eligibility criteria and were included in the series of systematic reviews of which this paper is a summary.

The 111 included studies were performed on 19,945 participants and used 69 different ambulatory assessment protocols or mood monitoring interventions. Table S1 in Multimedia Appendix 1 displays detailed characteristics of the studies and the ambulatory assessment protocols used. We included 34 qualitative studies, 28 RCTs, and 49 nonrandomized studies. Follow-up periods varied from 3 months to 3 years. All studies included participants with depression or BD; however, 10 studies used mixed samples [20-28]. Four studies used samples of young people or adolescents [29-32], while the rest used adult samples.

RCTs and nonrandomized studies used ambulatory assessment or mood tracking as either an outcome or an intervention. Qualitative studies explored the user experience of ambulatory assessment or mood tracking protocols or used coproduction or implementation frameworks to improve the usability and acceptability of the protocol. Table 1 outlines the overall findings from all systematic reviews.

The evidence-based recommendations from our research are summarized in Table 2, along with the strength of evidence supporting them, as assessed using GRADE [19].

Discussion

Recommendations for Future Development—Mood Monitoring Interventions

Future Research Focus

Future research should definitively establish whether mood monitoring interventions have benefits for mood or other outcomes by themselves in people with BD and depression. These interventions may be most successful in asymptomatic individuals with good insight who are at risk of relapse, although this decreased relapse risk for asymptomatic low-risk individuals was only found in 1 RCT [33]. These should be evaluated with an appropriate control group that allows the effectiveness of mood monitoring alone to be assessed, rather than mood monitoring in addition to many other components, such as cognitive behavioral therapy. The vast majority of trials used mood monitoring as an outcome assessment in both arms, and so we were unable to investigate the effectiveness of mood monitoring alone in these studies. There is a tension in our results: in the qualitative work, many participants report that they value self-monitoring and that it is effective for them, while the quantitative studies do not show an effect in reducing symptom severity, but it is possible that the benefits of mood monitoring are more indirect through more sense-making and empowerment to take action and have hope of mood improvement and may be further exacerbated via the Rosenthal effect. Trials in mood monitoring assume that symptom reduction is the goal of self-management and alternative models may include a focus on functioning and quality of life despite symptoms.

Mechanism of Action

This qualitative research suggests effectiveness and provides clear mechanisms for relapse prevention. We demonstrate that users record their mood to then guide a wide range of self-management strategies. Mood monitoring allows users to improve their insight into their condition and assess their mood objectively. Some of the self-management strategies that followed improved insight included lifestyle adjustments to then prevent relapse. Many of these self-management strategies were highly personalized and iteratively developed over time to work optimally for the individual, as they increasingly gained insight into their disorder. Some of these strategies reported by participants included secondary prevention strategies: establishing a dialog with trusted individuals about their mood and how to manage it (eg, trusted individuals, friends or family), considering how to improve their sleep, doing more exercise at the right times, and thinking carefully about additional interventions to stop their mood from worsening, for example, contacting a health care professional. These strategies generally combine the expertise of the experience of the patient with the clinical professionalism of the clinician to provide a richer picture of the context and care required. Any form of integrated care planning with professionals, however, will depend on IT system integration, which carries further risks around data security; therefore, the benefits would need to be clearly demonstrated.

Personalization

Because these self-management techniques are so variable, it would be very difficult to incorporate them into a mood monitoring intervention. The protocol, however, could represent a framework that offers data to then allow the user to use this new information to then deploy their own self-management strategies in a highly personalized way. This kind of personalization and flexibility was important to both people with depression and BD, and the intervention could be constructed to support this. For example, the technology used by the intervention could be less personalized, but the way the data are used might be more highly personalized—for example, in basic mood monitoring data being used in nuanced and iterative ways for self-management. As the technology becomes more personalized, it is likely that many people will wish for some functions to be disabled, as they do not consider them appropriate for themselves at that time. For example, users wanted different amounts of control and autonomy over the collection of their behavioral and mood data, such as toggling off certain functions that were deemed intrusive. This would also help to alleviate data privacy concerns, which were common, but not shared by all participants.

Interface With Mental Health Services

We studied how these interventions might communicate with and be supported by mental health services and if this was desired by people with depression or BD. We proposed three different options: (1) self-management or monitoring without any involvement of mental health services, (2) mood monitoring occasionally shared with mental health services, and (3) clinician-directed monitoring. Many users with depression felt that sharing the data with their clinician was essential for the purpose of the ambulatory assessment, for example, providing accountability, setting goals, and using the data to have a detailed discussion with their health care professional around management strategies. Many people with BD, however, expressed skepticism about the help and support they would receive from mental health services and did not wish for the intervention to be integrated into their management by these services, instead using it in a personal sphere to self-manage their condition. Mood monitoring involving trusted individuals or family members was, however, frequently mentioned by users but rarely incorporated into the protocol itself.

Customizability

We thus advise that customizability be fundamental to future protocol development and incorporated using the recommendations made in this paper to maximize user engagement and successful uptake, and there are good methodological justifications for this [34,35]. Improving the “fit” of the intervention is likely to improve engagement and outcome [36]. People with depression or BD wished for the following types of personalization: the type of data collected, the data shared, the feedback the protocol offers, and notifications or wearables. Participants wished for an intuitive and easy-to-use passive data protocol that builds on their existing strengths with a high emphasis on personalization and customizability. Adapting and digitizing existing strengths-based models may improve user experience [37,38].

Incorporating these measures may improve adherence and attrition and potentially address some adverse events, such as increases in stress and how confronting the process feels. Future protocols should test additional therapeutic elements in the hope of managing these adverse events, which may present a significant barrier to future uptake. Furthermore, co-design and the use of responsible research and innovation techniques are both ways to de-risk innovations and interventions [39].

The Ideal Mood Monitoring Intervention

In our view, the ideal mood monitoring intervention would be based on a wide variety of passive data from smartphone sensors and other wearables (these measures are evolving and may include complex audio-visual data analysis [40]). It would have good performance, and users could trust that the mood prediction was accurate. It may be that passive data alone are not good enough to predict mood changes, and users may need to combine these data with an established mood measure to confirm mood change if the passive data detect an early shift. The intervention would be customizable in the sense that users could toggle data on and off if one particular aspect was deemed intrusive. The data would be stored in a secure way, and data sharing is consented to by the user in advance. The intervention would not have any technological problems—these were frequently described by participants as a barrier to use. Passive data methods, however, are not always low burden, as some individuals found passive wearables burdensome (eg, wearing an uncomfortable smartwatch overnight that may interfere with sleep [17]). Even with the simplest active ambulatory assessment measures, such as single-item mood reporting, many individuals found these intrusive and burdensome, particularly when depressed. Ambulatory assessment protocols should therefore allow the personalization of data collection methods and frequency to account for differing perceptions of burden [41].

The actual intervention could be relatively simple—providing occasional and clear feedback when a user’s mood is deteriorating—but these data could be used in a far more nuanced and personalized way. For example, this can then allow the user to implement self-management strategies that they may have already developed and hopefully prevent relapse, for example, via a well-being, recovery, or crisis plan. The intervention could remind the user of their crisis plan, and the user could preselect things they wish to be reminded of when their mood deteriorates. The intervention could serve both to detect early warning signs of mood deterioration but also to retrospectively assess mood over the long term to allow for appraisal of their self-management strategies and provide hope that dips can be recovered with appropriate support. These data could also be shared to guide decision-making, for example, medication choices and assessing effectiveness with objective data. These self-management strategies may or may not include involvement with mental health services, and any wish for involvement should not be assumed.

Particularly for people with depression, the intervention should aim to incorporate therapeutic elements to address adverse events—in particular the potential for subjective worsening of mood and how confronting the protocol can be. Any feedback the app offers should be co-designed with people with lived experience of BD or depression, as we report some feedback mechanisms via protocols that participants felt were patronizing and were detrimental to use. Participants wished for use to feel positive and to build on their strengths—while acknowledging that this may not always be possible. This may demonstrate a preference for selective reflection, such as that used in other therapeutic techniques, for example, motivational interviewing—where there is a slight bias toward adopting healthier habits. Medication reminders were perceived positively. Some protocols may require some form of onboarding or preparation to use—this is to consider purpose and apply some of the potential uses of a mood monitoring tool to someone’s life in order for them to maximize its benefits. This could be with a clinician or with a digital navigator—someone who knows and understands both the technology but also the health challenges that people may have through their disability or ill-health [42,43]. The app would not replace clinician time but be used as an additional tool for people to self-manage their mental illness, or improve the efficiency of time spent with a clinician. Insight gained during tracking is a potential tool for shared decision-making and may therefore contribute toward patient activation.

Recommendations for Ambulatory Assessment in Research

Performance

The performance of these protocols was unclear, particularly when assessing passive measures of mood. The performance of active ambulatory assessment was good (correlation coefficients: moderate or strong strength), but this was often using already validated measures of mood, administered more frequently. Performance is a key issue for people with mood disorders, as it signals the degree of trust (eg, and thus adherence or attrition) they should place in any feedback that is provided, which in turn can cause them to adjust the management of their illness.

Standardization

This review highlights the need for possible standardization in ambulatory assessment methodologies (eg, in the frequency, duration, devices used, interventions delivered, association or validation) to support the implementation of these behavioral assessment measures. In depression, 423 different smartphone-based passive ambulatory assessment performance measures were extracted, but few overlapped across studies, and many lacked validation or comparative measures. Current performance, particularly with passive ambulatory assessment measures, demonstrates large methodological variation and inconsistent reporting. The wide range of ambulatory assessment protocols used across studies and the differences in reporting metrics severely limit the ability to compare performance outcomes. Standardized reporting frameworks may address some of these issues, ensuring findings can be reliably replicated and robustly compared.

It might not be plausible, however, to standardize ambulatory assessment protocols relatively quickly, as the depression measurement tools we identified here varied significantly—from established questionnaires to some simple question or questions and from a full scale to a couple of core items or even a single item. To compare the results from different studies with various depression measures, we propose the following approaches.

One method is to use standardized quantities such as the standardized mean difference or Cohen d, which were derived from different studies [44]. The disadvantages of this, however, are that scale-free standardized scores lack contextual meaning, are sensitive to variability, and have arbitrary interpretations [45,46]. Another possible method is to convert the scores between different depression questionnaires [47-49]. This method also has its limitations, and it ignores differences in the questionnaire’s content validity and structure. For example, the 17-item Hamilton Depression Rating Scale and the 9-item Patient Health Questionnaire measure different components of depression [50,51], and the underlying constructs of many different depression questionnaires differ slightly [52]. Another limitation is that the conversion does not reliably preserve severity‐category alignment (eg, mild, moderate, severe), and this can lead to mismatches in the clinical interpretation [53]. Considering the above methodological limitations and the widespread use of certain depression measures, we suggest using established depression measures that have sound psychometric characteristics in ambulatory assessment, and this must be guided by clinical need.

The Ideal Ambulatory Assessment Research Protocol

In our view, the ideal ambulatory assessment research protocol would be digital and rely principally on passive data but may need to confirm or validate any mood shift detected using passive data with a validated self-report measure, for example, the 9-item Patient Health Questionnaire. This is because passive data collection can only detect observable changes in behavior or physiological function; it cannot measure the subjective experience of mood that self-rated measures reveal. Digital methods allow for more customizability and personalization and are perceived as being easier to use. Any self-report measures should aim to be triggered at a time that is convenient for the participant, for example, self-selected in advance with an option to delay the assessment within a reasonable timeframe. This is because passive approaches are favored by many participants and may address concerns that active ambulatory assessment can be stressful and perceived as a burden, and active engagement can be a distressing reminder of one’s mental illness. It also overcomes issues with recall bias and helps participants with mood disorders who struggle to fill in assessments when they are more severely unwell, for example, due to memory, concentration, or motivation issues.

Ideally, the ambulatory assessment would not exert any effects on mood, which could obfuscate or confound the results in intervention studies. This, however, may be unrealistic (eg, Hawthorne effect), and it may be possible that the frequent assessment will either deteriorate or improve mood, especially if there is any feedback on the mood itself, and this information is used by the participant, consciously or unconsciously, to make adaptations in their life [54]. There would be notifications to complete the ambulatory assessment, but the user would have some control over their frequency. The ambulatory assessment should be carefully tested and free of major technological issues, which are a frequent barrier to use. Some participants reported that frequent crashes and syncing problems with wearables were their reasons for dropping out, increased the perceived burden of using the app, and undermined trust in the feedback provided by the app. Participants may require support prior to using the ambulatory assessment, for example, demonstrating use, what can be controlled by them (eg, notifications), what cannot be controlled. This is likely to be most effective when this initial support comes from a known or trusted person rather than a general tutorial or automated system. Again, the use of the ambulatory assessment protocol should not completely replace human contact, and participants still valued this. The initial assessment may benefit from being in person or performed by someone who the participant already has a trusting relationship with. This may also improve attrition or adherence, although this hypothesis requires further empirical evaluation. Researchers should be able to address data security concerns that are common and of concern to people with mood disorders.

Some of our analyses were limited by a paucity of reporting of performance measures, particularly those in forms that would be easily comparable between studies. Future research should report the performance of ambulatory assessment protocols in standardized ways in accordance with other areas of scientific research [55].

Strengths and Limitations of the Work

This paper builds on previous work to triangulate the findings from a wealth of quantitative and qualitative data exploring optimum development from a wide variety of perspectives. The recommendations themselves are drawn by the team that performed the systematic review bringing together psychiatrists, methodologists, and service users who would necessarily agree on their nature or importance. The findings therefore catalytic validity. The literature itself is drawn from a range of Western countries and not from low- and middle-income countries, which is an important gap in the research. The evidence was not put to an independent external interdisciplinary group of experts, and we did not use any consensus-drawing methodology such as Delphi. Therefore, the recommendations are not definitive but interim and suitable for a field where technological innovation is likely to quickly progress as artificial intelligence, wearables, bots, and avatars are developed and used to monitor, measure, and manage mood.

Conclusions

We assess ambulatory assessment protocols and mood monitoring interventions and provide clear evidence-based recommendations for future development. Future iterations of ambulatory assessment protocols and mood monitoring interventions should consider these findings and how they might incorporate some of these themes into development. Crucially, the requirements and recommendations are different depending on the purpose, for example, ambulatory assessment for research or digital mood monitoring intervention. There are, however, overlapping themes, such as personalization and performance. These recommendations have been formulated closely with people with lived experience, by reviewing qualitative research and having the results checked for salience and coherence by individuals with lived experience and clinicians. Future ambulatory assessment protocols must balance personalization, performance, and safety if they are to fulfill their substantial promise in both research and clinical practice. None of these measures are yet robust enough to replace current outcomes, but they can add additional granularity and confirmation (since they are free of recall bias and collected in the moment). There remains great potential for these measures if the detailed challenges can be overcome.